Selective outcome reporting

Edit 0 14 …

What is selective outcome reporting?


Selective outcome reporting is "the selection of a subset of the original variables recorded, on the basis of the results, for inclusion in publication of trials," [Hutton, 2000] and is relevant to reporting bias ("systematic differences between reported and unreported findings" [Higgins]). The primary concern is that results will be reported according to their statistical significance, with non-significant results being withheld. Other forms of selective outcome reporting that are relevant at the individual study level include the choice of data for an outcome (e.g., in the case of multiple scales being used, but not all reported), analysis of the data (e.g., converting a continuous to a dichotomous variable), selective reporting of subsets of the data, and under-reporting of data (e.g., stating merely that p>0.05).

Journal space constraints are a common concern in relation to selective outcome reporting, with authors worrying that they will be unable to report everything that they have included in their trial. Many journals, though, now publish additional material online, facilitating complete reporting of outcomes.

Tools


Appropriate means of reporting outcomes:
  • The study protocol is available and all of the study’s pre-specified (primary and secondary) outcomes that are of interest in the review have been reported in the pre-specified way
  • The study protocol is not available but it is clear that the published reports include all expected outcomes, including those that were pre-specified

Outcome measure planning tool: Use this to map out the outcomes you plan to measure and analyze

Trial Registration: Since July 1, 2005, the International Committee of Medical Journal Editors (ICMJE) has endorsed prospective trial registration (ICMJE Statement). This includes registering all trials with a publicly available registry prior to participant enrollment. Registries accepted by the ICMJE include:
  • Australian New Zealand Clinical Trials Registry (ANZCTR)
  • Brazilian Clinical Trials Registry (ReBec)
  • Chinese Clinical Trial Registry (ChiCTR)
  • Clinical Research Information Service (CRiS), Republic of Korea
  • Clinical Trials Registry - India (CTRI)
  • Cuban Public Registry of Clinical Trials (RPCEC)
  • EU Clinical Trials Register (EU-CTR)
  • German Clinical Trials Register (DRKS)
  • Iranian Registry of Clinical Trials (IRCT)
  • Japan's UMIN-CTR
  • Pan African Clinical Trial Registry (PACTR)
  • Sri Lanka Clinical Trials Registry (SLCTR)
  • The Netherlands' Trialregister.nl
  • The United States' ClinicalTrials.gov
  • The International ISRCTN.org

The WHO maintains the International Clinical Trials Registry Platform (ICTRP), a searchable database of multiple registries. It also provides a useful overview of trial registration, including:

1) Why is trial registration important?
2) How to register a trial
3) Organizations with policies

The registry can be searched at: http://apps.who.int/trialsearch/. A pediatric search filter can be applied by using the "Advanced Search" option and checking the box that says "Search for clinical trials in children."

[back to top]

Examples


On this page we've compiled a number of examples of risk of bias assessments - the good, the bad, and those that are a bit unclear. Feel free to work through them yourself and come up with an assessment of low, unclear, or high risk of bias (our judgments and rationale are on the assessments page), or download a spreadsheet file with the same information. RoB assessments are divided up into the seven major domains: sequence generation, allocation concelment, blinding of participants/personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting, and other sources of bias. A quotation is given with the article title following in brackets.

If you have other examples, please add them to the list!

Risk of Bias Guidelines

Download examples: external image vnd.openxmlformats-officedocument.spreadsheetml.sheet.png Examples of Risk of Bias Assessments.xlsx external image vnd.ms-excel.png Examples of Risk of Bias Assessments.xls




[back to top] [RoB Assessment Page]

References


  1. Hutton JL, Williamson PR. Bias in meta-analysis due to outcome variable selection within studies. Journal of the Royal Statistical Society Series C 2000;49:359-370.
  2. Ioannidis JP, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 2001; 285:437-443. [PubMed]
  3. Chan AW, Hróbjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457-2465. [PubMed]
  4. Chan AW, Krleza-Jeric K, Schmid I, et al. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004;171:735-740. [PubMed]
  5. Papanikolaou PN, Ioannidis JP. Availability of large-scale evidence on specific harms from systematic reviews of randomized trials. Am J Med 2004; 117:582-589. [PubMed]
  6. Von Elm E, Rollin A, Blumle A, et al. Selective reporting of outcomes of drug trials? Comparison of study protocols and published articles. 14th Cochrane Colloquium 2006;Oct 23-26 (Dublin, Ireland). [Cochrane]
  7. Dwan K, Altman DG, Amaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 2008;3:e3081. [PubMed]
  8. Mathieu S, Boutron I, Moher D, et al. Comparison of registered and published primary outcomes in randomised controlled trials. JAMA 2009;302:977-984. [PubMed]
  9. Ioannidis JP. Adverse events in randomized trials: neglected, restricted, distorted, and silenced. Arch Intern Med 2009; 169:1737-1739. [PubMed]

[back to top]